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This  article,  to  appear  in  AI  Magazine  (Summer,  1990),  reviews  work  done  in  the"area  of. 
Planning  Systems.  The  goal  of  this  article  is  to  familiarize  the  reader  with^oihe  of  the  impor¬ 
tant  problems  which  have  arisen  in  the  design  of  planning  systems,  arid  some  of  the  many 
solutions  which  have  been  developed  in  the  over  thirty  years  of  planning  research.  In  this  pa¬ 
per  we  provide  a  broad  coverage  of  the  major  ideas  in  the  field  of  ®  planning  and  show  the 
direction  in  which  some  current  research  is  going.  We  define  some  of  the  terms  commonly 
used  in  the  planning  literature,  describe  some  of  the  basic  issues  arising  out  of  the  design  of 
planning  systems,  and  survey  results  in  the  area.  As  such  a  task  is  virtually  never-ending,  and 
any  finite  document  must  per  force  be  incomplete,  we  also  provide  references  to  connect  each 
idea  to  the  appropriate  literature  and  to  allow  readers  access  to  the  papers  most  relevant  to  their 
own  research  or  applications  work. 
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1.  Introduction 

A  long  standing  problem  in  the  field  of  automated  reasoning  is  that  of  designing  systems  which 
can  describe  a  set  of  actions  (or  plan)  which  can  be  expected  to  allow  the  system  to  reach  a 
desired  goal.  Ideally,  the  set  of  actions  so  produced  is  then  passed  on  to  a  robot,  a  manufacturing 
system,  or  some  other  form  of  effector,  which  can  follow  the  plan  and  produce  the  desired  result. 
The  design  of  such  “planners”  has  been  with  AI  since  its  earliest  days,  and  a  large  number  of 
techniques  have  been  introduced  in  progressively  more  ambitious  systems  over  a  long  period.  In 
addition,  planning  research  has  introduced  many  problems  to  the  field  of  AI.  Some  examples 
include: 

•  Representing  and  reasoning  about  time,  causality  and  intentions 

•  Physical  or  other  constraints  on  suitable  solutions 

•  Uncertainty  in  the  execution  of  plans 

•  Sensation  and  perception  of  the  “real  world"  and  holding  beliefs  about  it 

•  Multiple  agents  who  may  cooperate  or  interfere 

Planning  problems,  like  most  AI  topics,  have  been  attacked  in  two  major  ways:  approaches  which 
try  to  understand  and  solve  the  general  problem  without  use  of  domain  specific  ’  -.owledge,  and 
approaches  which  use  domain  heuristics  directly.  In  planning,  these  are  often  referred  to  as 
domain-dependent  approaches  (which  use  domain-specific  heuristics  to  control  the  planner’s 
operation),  and  domain-independent  approaches  (in  which  the  planning  knowledge  representation 
and  algorithms  are  expected  to  work  for  a  reasonably  large  variety  of  application  domains).  The 
issues  involved  in  the  design  of  domain-dependent  planners  are  those  generally  found  in  applied 
approaches  to  AI  —  the  need  to  justify  solutions,  the  difficulty  of  knowledge  acquisition,  and  the 
fact  that  the  design  principles  may  not  map  well  from  one  application  domain  to  another. 


An  early  version  of  this  article  appeared  in  the  Knowledge  Engiv'-rng  Review  Volume  1  Number  2  pages  4-17, 
June  19RS  Part  of  that  r:.iew  along  with  a  tutorial  introduction  to  the  field  appeared  in  the  planning  chapter 
of  Knowledge  Engineering  Volume  1  published  by  McGraw-Hill  (editor  H.Adeli)  in  1990.  A  version  of  this  paper 
stressing  technical  definitions  appears  in  the  book  Readings  in  Planning,  (Allen,  Hendler,  and  Tate  eds.,  1990). 
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Figure  1:  A  brief  chronology  of  some  well-known  planning  systems. 

Numbers  in  parentheses  represent  systems  on  which  each  planner  has  built  directly  (also 
shown  as  solid  lines  where  possible).  Dotted  lines  represent  some  of  the  important  out* 
side  areas  influencing  the  development  of  planning  systems. 
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Planner 

Domain 

STRIPS  (Fikes  &  Nilsson,  1972a) 

HACKER  (Sussman,  1973) 

NOAH  (Sacerdoti,  1977) 

NONLIN  (Tate,  1977) 

NASL  (McDermott,  1978) 

OPM  (Hayes-Roth  &  Hayes-Roth,  1979) 
ISIS— II  (Fox  et.  al.,  1981) 

MOLGEN  (Stefik,  1981a) 

SIPE  (Wilkins,  1983) 

NONLIN+  (Tate  &  Whiter,  1984) 

DEVISER  (Vere,  1983) 

FORBIN  (Miller  et.  al.,  19851 

Simple  Robot  Control 

Simple  Program  Generation 

Mechanical  Engineers  apprentice  supervision 
Electricity  Turbine  Overhaul 

Electronic  Circuit  Design 

Journey  Planning 

Job  Shop  Scheduling  (Turbine  Production) 
Experiment  Planning  in  Molecular  Genetics 
Aircraft  Carrier  Mission  Planning 

Naval  Logistics 

Voyager  Spacecraft  Mission  Sequencing 

Factory  Control 

Table  1:  Numerous  planning  systems  have  attempted  to  integrate  available 
planning  techniques  and  apply  them  to  application  domains. 
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Figure  1  about  here. 


Work  in  domain-independent  planning  has  formed  the  bulk  of  the  AI  research  in  the  area  of 
planning.  The  long  history  of  these  efforts  (see  figure  1)  has  led  to  the  discovery  of  many  recur¬ 
ring  problems,  as  well  as  to  certain  standard  solutions.  In  addition,  there  have  been  a  number  of 
attempts  to  combine  the  planning  techniques  available  at  a  given  time  into  prototypes  able  to 
cope  with  increasingly  more  realistic  application  domains.  (Table  1  shows  an  example  of  some  of 
these  efforts  and  the  domains  to  which  they  were  applied.) 


Table  1  about  here. 


The  goal  of  this  article  is  to  familiarize  the  reader  with  some  of  the  important  problems  which 
have  arisen  in  the  design  of  planning  systems,  and  some  of  the  many  solutions  which  have  been 
developed  in  the  over  thirty  years  of  planning  research.  In  this  paper  we  will  try  to  provide  a 
broad  coverage  of  the  major  ideas  in  the  field  of  AI  planning  and  attempt  to  show  the  direction  in 
which  current  research  is  going.  We  will  define  some  of  the  terms  commonly  used  in  the  planning 
literature,  describe  some  of  the  basic  issues  arising  out  of  the  design  of  planning  systems,  and  sur¬ 
vey  results  in  the  area.  (Due  to  the  recurrence  of  many  themes  throughout  the  planning  litera¬ 
ture,  we  will  survey  the  field  on  the  basis  of  areas  of  interest  rather  than  in  terms  of  its  chrono¬ 
logical  development.)  Such  a  task  is  virtually  never-ending,  and  any  finite  document  must  per 
force  be  incomplete.  Thus,  in  addition  to  our  discussion  of  the  issues  we  have  provided  references 
to  connect  each  idea  to  the  appropriate  literature  and  to  allow  readers  access  to  the  papers  most 
relevant  to  their  own  research  or  applications  work. 


2.  Planning  Terminology 

An  AI  planning  system  is  charged  with  generating  a  plan  which  is  one  possible  solution  to  a 
specified  problem.  The  plan  generated  will  be  composed  out  of  operator  schemata,  provided  to 
the  system  for  each  domain  of  application.  This  section  briefly  considers  the  meaning  of  each  of 
these  terms  and  how  they  relate  to  one  another. 

A  problem  is  characterized  by  an  initial  state  and  goal  state  description.  The  in  al  state 
description  tells  the  planning  system  the  way  the  world  is  “right  now”.  The  goal  state  descrip¬ 
tion  tells  the  planning  system  the  way  we  want  the  world  to  look  when  the  plan  has  been  exe¬ 
cuted.  The  world  in  which  planning  takes  place  is  often  called  the  application  domain.  We  will 
sometimes  refer  to  the  goal  state  description  as  simply  “the  goal.”  In  many  systems,  a  goal  may 
be  transformed  into  a  set  of  other,  usually  simpler,  goals  called  “sub-goals.” 

Operator  schemata  characterize  actions.  (The  terms  action  and  event  are  often  used  interchange¬ 
ably  in  the  AI  planning  literature,  and  certainly  will  be  here.)  Schemata  primarily  describe 
actions  in  terms  of  their  preconditions  and  effects.  Plans  are  built  out  of  these  operator  sche¬ 
mata.  Each  operator  schemata  characterizes  a  class  of  possible  actions,  by  containing  a  set  of 
variables  which  can  be  replaced  by  constants  to  derive  operator  instances  that  describe  specific, 
individual,  actions.  When  the  distinction  doesn’t  matter,  we’ll  use  the  term  operator  to  stand  for 
both  operator  schemata  and  operator  instances.  An  action  which  the  planner  considers  to  be 
directly  executable  is  referred  to  as  a  primitive  action,  or  simply  as  a  primitive. 


Figure  2  about  here. 
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Pickup(x) 

Precondition: 

ONTABLE(x)  A 
HANDEMPTY  A 
CLEAR(x) 

Delete  List: 

ONTABLE(x) 

HANDEMPTY 

CLEAR(x) 

Add  List: 

HOLDING(x) 

Figure  2:  A  typical  STRIPS  operator. 

The  operator  Pick-up  contains  a  precondition  formula,  add  and  delete  lists.  Thus,  pick¬ 
up  can  be  used  to  lift  an  object  which  must  be  on  the  table,  the  hand  must  be  empty, 
and  the  object  must  be  clear  (from  the  preconditions).  After  pickup,  the  object  is  not  on 
the  table  and  is  not  clear,  and  the  hand  is  no  longer  empty  (from  the  delete-list),  also  the 
hand  is  known  to  be  holding  the  object  (from  the  add-list). 


A  commonly  used  terminology  throughout  the  AI  planning  literature  is  that  of  STRIPS  operators 
(Fikes,  et.al.,  1972a;  1972b).  These  operators,  first  used  in  the  early  planning  program  STRIPS, 
describe  an  action  by  means  of  three  parts:  a  precondition  formula,  an  add-list  and  a  delete-list 
(see  Figure  2).  An  operator’s  precondition  formula  (or  simply,  the  operator’s  preconditions)  give 
facts  that  must  hold  before  the  operator  can  be  applied.  The  add-list  and  delete-list  are  used  in 
concert  to  simulate  action  occurrence.  If  an  operator’s  preconditions  hold  in  a  state  then  the 
operator  can  be  applied.  Applying  an  operator  means  acting  on  its  add-list  and  delete-list  to 
produce  a  new  state.  The  new  state  is  produced  by  first  deleting  all  formulas  given  in  the 
delete-list  and  then  adding  all  formulas  in  the  add-list.  Although  newer  planning  systems  (dis¬ 
cussed  later)  do  depart  from  this  approach,  the  terminology  of  STRIPS  operators  is  fairly  stan¬ 
dard  and  we  will  use  it  often  in  this  article. 

A  plan  is  an  organized  collection  of  operators.  A  plan  is  said  to  be  a  solution  to  a  given  problem 
if  the  plan  is  applicable  in  the  problem’s  initial  state,  and  if  after  plan  execution,  the  goal  is  true. 
What  does  it  mean  for  a  plan  to  be  “applicable”?  Assume  that  there  is  some  operator  in  the  plan 
that  must  be  executed  first.  Then  the  plan  is  applicable  if  all  the  preconditions  for  the  execution 
of  this  first  operator  hold  in  the  initial  state.  Repeated  analysis  can  determine  whether  or  not  all 
operators  can  be  applied  in  the  order  specified  by  the  plan.  This  analysis  is  referred  to  as  tem¬ 
poral  projection.  The  first  state  considered  in  the  projection  is  the  problem’s  initial  state. 
Repeated  operator  applications  produce  intermediate  state  descriptions.  If  the  goal  is  true  at  the 
end  of  this  projection  then  the  plan  is  a  solution  to  the  specified  problem. 

So,  the  inputs  to  a  typical  AJ  planning  system  are  a  set  of  operator  schemata  and  a  problem  that 
is  characterized  by  an  initial  state  description  and  goal.  The  output  from  the  planner  is  a  plan 
which  under  projection  satisfies  the  goal.  The  process  connecting  the  input  and  output  is  known 
by  various  names.  Common  names  are  plan  generation,  plan  synthesis  and  plan  construction.  A 
planner  is  called  domain  independent  in  the  sense  that  the  plan  representation  language  and  plan 
generation  (or  synthesis,  or  construction)  algorithms  are  expected  to  work  for  a  reasonably  large 
variety  of  application  domains.  While  this  view  of  planning  is  slightly  restrictive,  it  will  suffice 
for  the  purposes  of  this  overview. 

A  planner  is  organized  so  that  it  defines  a  search  space  and  then  seeks  a  point  in  that  search 
space  which  is  defined  as  a  solution.  Planners  differ  as  to  how  they  define  their  search  space. 
Some  (most  of  the  pre-1975  planners)  define  points  in  the  search  space  as  "states"  of  the 
application’s  world  at  various  times.  This  "world  state"  can  be  traversed  by  applying  any  appli¬ 
cable  operator  to  some  chosen  state  leading  to  a  different  point  in  the  search  space.  A  problem 
solution  can  be  defined  as  a  sequence  of  operators  that  can  traverse  the  search  space  from  some 
initial  state  to  some  state  defined  as  satisfying  the  goal.  Thus,  a  state-space  plan  contains 
descriptions  of  states  of  the  world  the  plan  is  to  be  executed  in,  and  move  operators  which 
correspond  to  the  actions  to  be  carried  out  to  execute  the  plan.  A  state-space  solution  is  trivial 
to  recognize:  it  is  quite  simply  a  path  from  some  initial  state  to  a  state  satisfying  the  goal 
specification  (see  Figure  3a). 

Other  (most  of  the  post-1975)  planners  define  points  in  the  search  space  as  partially  elaborated 
plans.  One  point  in  this  space  is  changed  into  another  using  any  applicable  planning  transforma¬ 
tion  such  as  the  expansion  of  an  action  to  a  greater  level  of  detail,  the  inclusion  of  an  additional 
ordering  constraint  between  actions  to  resolve  some  interaction  between  effects  of  unordered 
actions,  etc..  Given  some  initial  (skeleton)  plan  defining  a  point  in  this  “partial  plan  search 
space",  a  sequence  of  plan  transformations  must  be  applied  which  lead  to  a  fully  detailed  plan 
that  satisfies  the  goal  (see  Figure  3b).  A  plan  is  considered  to  be  complete  when  all  of  the  goals 
can  be  realized  as  a  set  of  primitive  actions  (totally-  or  partially -ordered  depending  on  the 
planner). 
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Figure  3:  Partial  solutions  to  the  blocks  world  problem  “(ON  A  B)  &  (ON  B 
C)” 

Figure  3a  (based  on  STRIPS)  shows  a  typical  state-space  and  a  sequence  of  operators, 
representing  a  partial  solution.  Figure  3b  (based  on  NOAH)  shows  the  plan  as  a  partial 
ordering  of  actions  representing  a  partial  solution. 


Figure  3(a,b)  about  here. 


Systems  which  search  through  the  space  of  partial  plans  have  typically  represented  plans  as  an 
“action-ordering”  in  which  the  actions,  described  by  operators,  are  strung  together  with  temporal 
ordering  relations.  The  most  famous  example  of  such  plans  are  Sacerdoti’s  (1975)  Procedural 
Nets  (see  section  3).  Action-ordering  plans  describe  the  relationships  among  actions  directly, 
rather  than  through  states  and  predicates  contained  within  states. 

In  an  action-ordering  representation  a  plan  need  not  completely  specify  the  conditions  that  each 
action  affects.  The  plan  can  simply  order  two  actions  without  specifying  intermediate  states.  In 
contrast,  state-based  plan  structures  typically  require  complete  specification  of  these  intermediate 
states.  This  is  particularly  difficult  for  use  in  describing  complicated  causal  and  temporal  rela¬ 
tionships  between  actions.  Thus,  many  complex  domains  are  quite  difficult  to  encode  using  a 
state-based  approach  (cf.  Lansky,  1987),  and  action-ordering  approaches  have  become  the  more 
generally  used  technique. 


3.  Planning  and  Search 

As  can  be  seen  from  the  discussion  above,  planning  is  essentially  a  search  problem.  The  program 
must  traverse  a  potentially  large  search  space  and  find  a  plan  which  is  applicable  in  the  initial 
state  and  produces  a  solution  to  the  goal  when  run.  This  search  can  be  quite  difficult  as  the 
search  space  may  contain  many  interactions  between  different  states  or  partial  plans.  These 
interactions  lead  to  a  surprising  amount  of  complexity  —  for  example,  establishing  the  existence 
of  a  precondition  in  a  partially  ordered  plan  may  require  exponential  computation  (Chapman, 
1987)  and  the  problem  of  finding  an  optimal  plan  in  even  a,  simple  Blocks  World  domain  has 
recently  been  shown  to  be  NP-hard  (Gupta  &  Nau,  1990).'  The  problems  of  organizing  the 
heuristic  search,  choosing  what  to  do  in  cases  of  failure,  and  generally  finding  ways  of  making 
more  informed  choices  have  therefore  been  among  the  most  discussed  in  the  planning  literature. 
(A  good  quantitative  discussion  of  planning  viewed  as  heuristic  search  can  be  found  in  Korf 
(1987).) 

Early  approaches  to  planning  sought  to  apply  legal  "moves*  to  some  initial  state  to  search  for  a 
state  that  satisfied  the  given  goals.  There  was  a  great  deal  of  overlap  with  game  playing  work. 
Heuristic  evaluation  functions  were  employed  to  rate  the  various  intermediate  states  produced  to 
estimate  their  "closeness"  to  a  goal  state  (e.g.  in  A*  (Hart  et  al,  1968)  and  the  Graph  Traverser 
(Doran  &  Michie,  1966)).  This  approach,  however,  was  found  wanting  due  to  the  difficulty  in 
designing  working  heuristics  and  the  usually  exponential  growth  of  the  search  space  . 

To  reduce  the  number  of  intermediate  states  considered,  Newell  and  Simon  (1963)  introduced 
“means-ends  anlysis”,  a  heuristic  which  involves  considering  only  those  activities  that  could 
satisfy  some  outstanding  goal.  Operators  which  when  run,  would  cause  the  current  state  to  more 
closely  resemble  the  goal  state,  were  preferred.  This  technique  was  used  as  the  basis  of  the  search 
in  many  of  the  early  planners  which  used  state-space-based  plans. 

With  the  introduction  of  Procedural  Nets  in  NOAH  (Sacerdoti,  1975),  the  search  problem 
changed.  In  this  system  and  its  descendants,  the  search  space  consists  not  of  a  set  of  ‘world 
states,’  but  rather  a  space  of  partial  plans.  For  any  non-primitive  action  in  the  current  network 
the  planner  may  consider  any  known  method  of  “reducing”  that  action  to  a  set  of  other  actions 
and/or  primitives.  Planning  in  such  a  system  consists  of  choosing  appropriate  reductions  from 

t  Planning  involving  conditional  effects  has  been  shown  to  be  undecidable(Chapman,  1987)  —  that  is,  a  plan 
generated  in  such  domains  cannot  be  guaranteed  to  succeed  without  some  type  of  execution  time  additions. 

This  issue  is  discussed  in  Section  6. 
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among  the  sets  of  possibilities  and  ordering  actions  so  as  to  eliminate  harmful  interactions. 


One  important  technique  introduced  for  searching  partial  plans  was  that  of  least  commitment 
plan  representations'  .  Such  representations  are  used  to  allow  a  set  of  plans  to  be  represented  in  a 
single  state  of  the  search.  Examples  of  such  representations  include  the  use  of  a  partially-ordered 
plan  to  represent  a  number  of  possible  action  orderings  prior  to  a  commitment  becoming  neces¬ 
sary  (e.g.  in  NOAH,  Sacerdoti  1975),  or  the  posting  of  constraints  on  objects  referred  to  in  the 
plan  rather  than  making  an  arbitrary  selection  (e  g.  in  MOLGEN,  Stefik  1981a). 

To  search  through  the  space  of  partially-ordered  plans  many  solutions  have  been  offered.  Some 
systems  do  not  search  through  the  possible  alternatives  at  all.  Instead,  selections  are  made  on  the 
basis  of  locally  available  information  and  a  commitment  is  made  to  a  particular  solution  path. 
(Of  course,  this  means  that  some  problems  cannot  be  solved.)  This  technique,  has  been  most  suc¬ 
cessful  where  strong  domain  heuristics  can  be  used  for  making  the  choices.  Where  such  heuristics 
are  not  available,  a  more  general  solution  is  to  use  backtracking  to  allow  backing-up  to  occur 
when  the  goal  cannot  be  reached  based  on  some  earlier  choice.  The  planning  system  simply  saves 
the  state  of  the  solution  at  each  point  at  which  there  are  alternative  ways  to  proceed  and  keeps  a 
record  of  the  alternative  choices.  The  first  is  chosen  and  search  continues.  If  there  is  any  failure, 
the  saved  state  at  the  last  choice  point  is  restored  and  the  next  alternative  taken  (if  there  are 
none,  “backtracking”  continues  over  previous  decisions).  Simple  stack  based  implementation  tech¬ 
niques  can  be  used  for  this  process  (for  example,  as  done  in  Prolog). 

The  NONLIN  program  (Tate,  1977)  introduced  a  variant  of  depth-first  backtracking  for  use  in 
planning.  Since  there  is  often  good  local  information  available  to  indicate  the  preferred  solution 
path,  it  is  often  appropriate  to  try  the  best  choice  indicated  by  local  heuristic  information  before 
considering  the  many  alternatives  that  may  be  available  should  the  local  choice  prove  faulty. 
Taken  to  the  extreme,  depth-first  search  gives  something  of  the  flavor  of  such  a  search  strategy. 
However,  gradual  wandering  from  a  valid  solution  path  could  entail  backtracking  through  many 
levels  when  a  failure  is  detected.  An  alternative  is  to  focus  on  the  choice  currently  being  made 
and  try  to  select  one  of  the  local  choices  which  seems  most  promising.  This  continues  while  all  is 
going  well  (perhaps  with  some  cut-off  points  to  take  a  long,  hard  look  at  how  well  things  are 
going).  However,  if  a  failure  occurs,  NONLIN  considers  the  entire  set  of  alternatives  which  have 
been  generated  (and  ranked  by  a  heuristic  evaluator). 

This  basic  technique  has  been  further  refined  to  provide  the  most  widely  used  approach  to  con¬ 
trolling  search  in  planning.  Any  backtracking  system  based  on  saved  states  and  resumption 
points  (whether  depth-first  or  heuristically  controlled)  can  waste  much  valuable  search  effort  — 
there  may  be  several  unrelated  parts  to  a  solution,  ’f  it  so  happens  that  backtracking  on  one  part 
has  to  go  back  beyond  points  at  which  work  was  done  on  an  unrelated  part,  all  effort  on  the 
unrelated  part  will  be  lost.  Many  planning  systems  avoid  this  problem  by  using  a  variant  on  the 
backtracking  methods  used  in  NONLIN.  These  systems  do  not  keep  saved  states  of  the  solution 
at  choice  points.  Instead,  they  record  the  dependencies  between  decision^,  the  assumptions  on 
which  they  are  based,  and  the  alternatives  from  which  a  selection  can  be  made.  They  then  use 
methods  for  undoing  a  failure  by  propagating  and  undoing  all  of  the  dependent  parts  of  the  solu¬ 
tion.  This  leaves  unrelated  parts  intact  irrespective  of  whether  they  were  worked  on  after  some 
undone  part  of  the  solution.  Examples  of  work  using  this  technique  include  Hayes  (1975),  Stall- 
man  and  Sussman  (1977),  NONLIN  f Decision  Graph  (Daniel,  1983)  and  MOLGEN  (Stefik. 
1981a, b)  to  some  extent. 


One  alternative  that  has  been  suggested  to  the  use  of  backtracking  based  search  is  that  of 
“opportunistic”  planning  systems.  These  systems  do  not  take  a  fixed  (goal  driven  or  data- 

t  Some  people  use  the  term  “least  commitment"  to  refer  only  to  the  ordering  of  plan  steps  in  a  partial-order 
planner.  We  use  the  term  in  the  broader  sense,  refering  to  any  aspect  of  a  planner  which  only  commits  to  a  par¬ 
ticular  choice  when  forced  by  some  constraints. 
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driven)  approach  to  solving  a  problem.  Instead,  a  current  “focus”  for  the  search  is  identified  on 
the  basis  of  the  most  constrained  opt,  ation  that  can  be  performed.  This  may  be  suggested  by 
comparison  of  the  current  goals  with  the  initial  weld  model  state,  by  consideration  of  the 
number  of  likely  outcomes  of  making  a  selection,  by  the  degree  to  which  goals  are  instantiated, 
etc.  Any  problem  solving  component  may  summarize  its  requirements  for  the  solution  as  con¬ 
straints  on  possible  solutions  or  restrictions  of  the  values  of  variables  representing  objects  being 
manipulated.  They  can  then  suspend  their  operation  until  further  information  becomes  available 
on  which  a  more  definite  choice  can  be  made  (e.g.  in  MOLGEN,  Stefik  1981a). 

There  are  a  number  of  planning  systems  which  have  an  operator-like  representation  of  the  dif¬ 
ferent  types  of  plan  transformations  available  to  the  planner.  A  separate  search  is  made  to  decide 
which  of  these  is  best  applied  <it  any  point;  this  happens  before  decisions  are  taken  about  the 
details  of  the  particular  application  plan  being  produced  (e.g.,  MOLGEN,  Stefik  1981b;  Wilensky 
1981a;  OPM,  Hayes-Roth  &  Hayes-Roth  1979;  PRS,  Georgeff  &  Lansky  1987).  This  technique, 
primarily  used  in  conjunction  with  opportunistic  planning,  is  often  refereed  to  as  “metaplanning" 
as  it  requires  the  planner  to  reason  not  only  about  the  goal,  but  also  about  the  various  techniques 
available  for  generating  the  plan. 


Besides  the  basic  method  of  reducing  the  search  space  by  selection  of  relevant  operators,  many 
other  methods  have  also  been  employed  in  planners.  Some  examples  include  Giving  goals  levels  of 
priority  and  considering  the  highest  priority  goals  first  (cf.  ABSTRDPS,  Sacerdoti  1973; 
LAWALY,  Siklossy  &  Dreussi  1975)  or  rejecting  states  or  p!u.,iS  that  are  known  to  be  impossible 
or  in  violation  of  some  rule  (cf.  WARPLAN,  Warren,  1974).  (This  latter  has  been  further  refined 
to  include  the  use  of  domain  constraints  (Allen  and  Koomen,  1983),  and  temporal  coherence 
(Drummond  &  Currie  1988,  1989)  to  provide  heuristics  for  rejecting  possible  actions).  A  variant 
o.:  this  approach,  using  a  domain  model  to  simulate  the  results  of  planning  operators  was  used  in 
Simmons  &  Davis’  (1987)  Generate-Test-Debug  planner.  DEVISER  (Vere,  1983),  SIPE  (Wilkins, 
1983),  NONLIN+  (Tate  &  Whiter,  1984)  and  0-Plan  (Currie  &  Tate,  1985)  use  checks  on 
resource  usage  levels,  time  constraints  on  actions,  and  other  resource  bounds  to  eliminate  some 
possibilities.  A  more  recent  approach  has  involved  the  use  of  parallelism  to  briefly  examine  many 
potential  choices  concurrently.  Heuristics  are  used  which  allow  the  effects  of  various  interactions 
detected  by  the  parallel  search  to  suggest  possible  choices  or  to  rule  out  potential  plans  (cf. 
SCRAPS,  Hendler  1987).  Another  approach  is  the  use  of  natural  occuring  “locality”  which  may 
occur  in  a  domain  to  reduce  the  search  burden  by  partitioning  the  planning  search  space  into 
smaller,  localized  search  spaces  (cf.  GEMPLAN,  Lanksy  1988). 


4.  Conjunctive  Goal  Planning 

In  addition  to  the  problem  of  controlling  the  search,  the  order  in  which  several  simultaneous  goals 
are  tackled  can  have  a  marked  effect  on  the  efficiency  of  the  search  process.  Some  early  planners, 
for  example,  could  loop  around  on  the  same  goals  repeatedly  or  get  redundant  solution  (in  the 
sense  that  the  final  plan  contained  steps  which  could  be  removed  without  negating  the  plan’s 
reaching  of  the  goal)  when  the  goals  were  attempted  in  the  wrong  order.  The  solving  of  such 
“conjunctive  goal”  plans  has  been  the  basis  of  much  of  the  modern  planning  research.  Two 
somewhat  orthogonal  approaches  have  been  taken  to  dealing  with  this  problem:  ordering  the  vari¬ 
ous  goals  by  levels  of  importance,  and  analyzing  and  avoiding  the  interactions  caused  by  interac¬ 
tions  between  conjunctive  goals. 


The  use  of  levels  to  partially  overcome  this  problem  (it  is  not  a  complete  solution)  was  intro¬ 
duced  in  ABSTRIPS  (Sacerdoti,  1973)  and  LAWALY  (Siklossy  &  Dreussi,  1975).  These  systems 
separated  the  goals  into  levels  of  importance  or  priority  with  the  more  abstract  and  general  goals 
being  worked  on  first  and  the  more  concrete  or  detailed  levels  being  filled  in  later.  A  solution  was 
formed  at  the  most  abstract  level  and  the  lower,  more  detailed,  levels  were  then  planned  using  the 
pre-set  skeleton  plan  formed  at  the  upper  levels.  No  backtracking  *o  the  higher  levels  was 
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possible.  Later  systems  (e.g.,  NONLIN,  Tate  1977)  treated  the  abstraction  levels  as  a  guide  to  a 
skeleton  solution,  but  were  able  to  re-plan  or  consider  alternatives  at  any  level  if  a  solution  could 
not  be  found,  or  if  it  was  indicated  that  a  higher  level  choice  was  faulty. 

Other  hierarchical  systems  use  the  abstraction  levels  as  one  guide  to  the  ordering  of  goals,  but 
have  other  mechanisms  that  can  be  considered  alongside  this.  Some  systems  are  able  to  deter¬ 
mine  when  a  particular  choice  (at  whatever  level)  is  sufficiently  constrained  to  be  a  preferable 
goal  to  work  on  at  any  time  (e.g.,  MOLGEN,  Stefik  1981a, b).  A  relatively  recent  approach  has 
included  attempting  to  build  models  which  can  plan  at  different  levels  of  the  hierarchy  con¬ 
currently.  A  mathematical  analysis  of  some  properties  which  allow  systems  to  assume  indepen¬ 
dence  of  level  effects  in  some  cases  has  been  performed  (Yang,  1989). 

A  second  aspect  of  handling  conjunctive  goal  planning  is  the  treatment  of  the  interactions  that 
arise  between  the  different  goals  in  the  planning  problem.  As  an  example,  consider  the  situation 
(from  Sacerdoti,  1977)  of  trying  to  paint  the  ceiling  and  also  painting  the  ladder  used  for  painting 
the  ceiling.  If  the  planner  paints  the  ladder  first,  the  ladder  will  be  wet  and  the  execution  agent 
will  be  unable  to  paint  the  ceiling.  Further,  getting  the  paint  for  the  ladder  and  getting  the  paint 
for  the  ceiling  should  be  combined  into  a  single  task.  Thus,  to  get  an  optimal  solution  the 
planner  must  partially  merge  the  two  separate  plans  (see  Figure  4). 


Figure  4  goes  about  here. 


Planners  can  be  categorized  according  to  the  way  in  which  they  manage  interactions  between 
goals  and  the  way  in  which  they  structure  operators  in  a  plan.  In  terms  of  goal  interactions, 
there  are  planners  which  make  the  so-called  “linearity  assumption”,  that  is,  that  solving  one  goal 
(out  of  a  conjunction)  and  then  following  this  with  the  solution  to  the  other  goals  in  the  conjunc¬ 
tion  will  be  successful.  This  assumption  is  valid  in  some  domains  since  the  solutions  to  different 
goals  may  often  be  decoupled.  Most  current  planners  do  not  make  the  linearity  assumption  and 
can  consider  arbitrary  interleavings  of  all  goals  and  subgoals.  These  planners  are  often  referred 
to  as  “non-linear”  planners’  as  they  do  not  require  the  linearity  assumption  to  guarantee  correct 
solution  T . 

Early  planning  systems  would  solve  the  conjunctive  goals  sequentially  and  then  make  a  simple 
check  to  see  if  the  conjunction  of  goals  still  held  (e.g.,  in  STRIPS  (Fikes  &  Nilsson,  1971).  Where 
the  linearity  assumption  failed,  this  could  lead  to  mutual  goal  or  subgoal  interference  which 
would  often  require  redundant  actions  being  put  into  the  plan.  In  fact,  in  the  worst  case  the 
planner  could  get  into  an  endless  cycle  of  re-introducing  and  trying  to  satisfy  the  same  goal  over 
and  over  again. 

Several  systems  followed  this  which  could  handle  some  of  the  problems  introduced  when  the  prob¬ 
lems  being  solved  involved  interacting  goals.  These  systems  either  allowed  alternate  orderings  of 
entire  solution  paths  (HACKER,  Sussman  1977),  allowing  an  interfering  action  (just  being  intro¬ 
duced  to  satisfy  some  goal)  to  be  placed  at  different  (earlier)  points  in  the  current  plan  until  a 
position  was  found  where  there  is  no  interaction  (WARPLAN,  Warren  1974)  or  regressing  the 
goal  (i.e.  considering  it  earlier)  when  an  interaction  with  some  particular  solution  failed  (Wald- 
inger,  1975).  (This  latter  had  the  advantage  that  redundant  actions  were  not  reintroduced  if  the 

t  The  term  "non-linear”  is  sometimes  used,  confusingly,  to  refer  not  to  this  assumption,  but  rather  to  the 
partial  ordering  of  plan  steps  that  some  planners  use  in  attempting  to  handle  the  problem.  We  address  this  issue 
later  in  this  section. 
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Paint  the  Ceiling  and  Paint  the  Ladder 


Paint  the  ceiling 

SPLIT 

1— .  Paint  the  ladder 


JOIN 


Get  paint  —  Get  Ladder  —  Put  paint  on  ceiling 


Get  paint 


Put  paint  on  ladder 


I 

JOIN 


SPLIT 


Get  paint  —  Get  Ladder 


Get  paint- 


JOIN. 


Put  paint  on  ceiling 

_ I 


Put  paint  on  ladder 


p—  Get  Ladder—  JOIN - 

SPLIT 

1— Get  paint—  JOIN. 


Put  paint  on  ceiling 


Put  paint  on  ladder 


Figure  4:  Painting  the  ceiling  (Based  on  Sacerdoti,  1977). 

(A)  The  conjunctive  goal  (B)  is  split  into  two  separate  tasks.  (C)  Plans  for  each  task  are 
generated  separately  and  (D)  ordering  links  are  introduced  to  avoid  interactions.  Finally, 
(E)  common  subgoals  are  merged. 


goal  was  achievable  earlier  in  the  plan). 


INTERPLAN  (Tate,  1975a)  introduced  a  representation  called  “Goal  Structure”  to  record  the 
link  between  an  effect  of  one  action  that  was  a  precondition  (subgoal)  of  a  later  one.  This 
representation  was  additional  to  the  ordering  links  between  the  actions  themselves  (as  some 
actions  have  effects  that  are  used  much  later  in  the  plan  —  Sussman  (1973)  referred  to  this  as  the 
plan’s  “teleology.”)  Interactions  were  detected  as  an  interference  between  some  new  action  or  goal 
being  introduced  and  one  or  more  previous  goal  structures.  A  minimum  set  of  goalreorderings 
could  be  suggested  that  corrected  for  the  interactions  found.  This  approach  was  found  to  be 
more  efficient  as  it  considered  fewer  re-orderings  of  the  given  goals  and  subgoais  than  the  earlier 
methods. 

NOAH  (Sacerdoti,  1975),  introduced  code  called  “critics"  which  were  used  to  search  for  interac¬ 
tions  between  parts  of  the  plan,  which  was  represented  as  a  partial  ordering.  This  was  an  impor¬ 
tant  innovation  as  it  allowed  the  planner  to  use  a  “least  commitment”  strategy  —  separate 
operators  would  be  considered  unordered  with  respect  to  each  other  unless  a  critic  introduced  an 
ordering  relation  to  avoid  an  interaction  (as  was  done  in  Figure  4d).  NOAH  used  a  data  structure, 
the  table  of  Multiple  Effects  (TOME),  to  record  the  effects  of  the  operators  and  thus  aid  in  dis¬ 
covering  interactions. 

NOAH  (Sacerdoti,  1977)  is  often  referred  to  as  the  first  “non-linear”  planner - the  term  being 

used  in  this  context  to  refer  to  the  partially  ordered  plan  structures  employed  by  the  system. 
This  is  often  confused  with  the  fact  that  NOAH  was  able  to  solve  problems  where  the  linearity 
assumption  didn’t  hold.  To  help  avoid  this  confusion  we  will  not  refer  to  this  type  of  system  as 
non-linear,  but  rather,  since  such  planners  use  partially  ordered  plans  we  will  use  the  term  “par¬ 
tial  order  planners.”  Thus,  NOAH  was  the  first  partial-order  planner. 

NONLIN  (Tate,  1975b;  1977)  was  a  partial-order  planner  modeled  on  NOAH.  As  in  NOAH,  the 
minimum  set  of  orderings  necessary  to  resolve  interactions  could  be  suggested  by  the  introduction 
of  ordering  links  into  a  partial  order  plan  when  they  became  essential.  The  NONLIN  system, 
however,  could  also  consider  alternatives  if  failures  on  any  chosen  search  branch  occurred.  NON¬ 
LIN  introduced  the  notion  of  a  Goal-State  Table  (GOST)  which  was  a  datastructure  which  could 
be  used  to  record  dependencies  between  plan  steps  facilitating  the  analysis  of  interactions.  Simi¬ 
lar  analyses  of  a  representation  of  the  effect  and  condition  structure  of  a  plan  to  detect  and 
correct  for  interactions  have  been  included  in  PLANX-10  (Sridharan  &  Bresina,  1985),  SIPE’s 
“Plan  Rationale"  (Wilkins,  1983)  and  Dean’s  (1985)  work. 

A  further  refinement  of  the  analysis  of  interactions  was  added  to  the  DEVISER  system  (Vere, 
1983).  When  individual  goals  or  actions  in  a  plan  had  time  constraints  on  when  they  could  be 
satisfied  or  performed,  the  detection  and  correction  of  interactions  were  sensitive  to  the  temporal 
displacement  introduced  .  This  helped  limit  the  number  of  legal  solutions  that  could  be  proposed. 
DEVISER  allowed  for  planning  in  the  presence  of  deadlines  and  external  events.  In  addition,  the 
use  of  objects  being  manipulated  as  scarce  resources  on  which  usage  conflicts  occur  and  need  to  be 
avoided  was  incorporated  in  the  MOLGEN  (Stefik,  1981a)  and  SIPE  (Wilkins,  1983)  planners. 
Viewing  time  as  such  a  resource  is  an  active  topic  being  considered  in  current  research  (cf.  Dean 
&  Boddy,  1987). 


5.  Operator  Representation 

The  very  first  systems  to  handle  planning  problems  simply  selected  appropriate  operators  to 
apply  to  a  problem  by  considering  the  “differences"  between  the  goal  state  and  the  initial  state 
and  looking  for  operators  which  were  known  to  remove  such  differences.  GPS  (Newell  &  Simon, 
1963),  for  example,  directly  associated  the  operators  for  the  problem  with  the  "differences"  they 
could  reduce.  Thus,  an  operator  such  as  “PAINT(Robot,x)”  would  be  associated  with  achieving 
the  fact  “PAINTED(x).” 
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STRIPS  (Fikes  and  Nilsson,  1971)  used  this  notion  of  differences,  as  well  as  ideas  from  the  situa¬ 
tion  calculus  (McCarthy  &  Hayes  1969)  and  Green’s  (1969)  QA3  program,  to  make  the  assump¬ 
tion  that  the  initial  world  model  would  only  be  changed  by  a  set  of  additions  to  and  deletions 
from  the  statements  modelling  the  world  state  —  everything  else  remaining  unchanged.  (This  is 
sometimes  called  the  "STRIPS  assumption.*)  STRIPS  then  defined  an  operator  as  having  an 
add-list,  a  delete-list  and  a  preconditions-formula  (to  define  the  applicability  or  sub-goaling  con¬ 
ditions).  Operators  were  selected  on  the  basis  of  the  recognition  of  goals  which  matched  state¬ 
ments  on  the  add-lists  (i.e.  those  statements  the  operator  could  add  to  the  current  state). 

For  non-primitive  actions,  Sacerdoti’s  (1975)  NOAH  system  used  procedurally  specified  "SOUP" 
(Semantics  Of  User’s  Problem)  code  to  introduce  appropriate  methods  of  achieving  goals  or  order¬ 
ings  to  correct  for  interactions  into  the  network  of  actions.  Later  planners  introduced  declarative 
representations  to  handle  this,  with  operators  which  were  extensions  to  the  STRIPS  operator  type 
of  formalism  (e.g.,  NONLIN’S  Task  Formalism  (Tate,  1977)  and  SIPE  Notation  (Wilkins,  1983)). 
As  well  as  add  and  delete  lists  and  precondition  formulae,  an  "expansion"  of  the  operator  to  a 
lower  level  of  detail  could  be  specified  as  a  partial  order  on  suitable  sub-actions  and  sub-goals. 

More  recent  systems  have  continued  to  add  more  information  to  the  operators.  One,  of  particular 
importance,  has  been  adding  information  about  resource  usage  to  the  operators,  so  that  planners 
can  reason  about  limited  resources.  Time  constraints  have  also  been  encoded  on  operators. 
DEVISER  (Vere,  1983),  for  example,  provided  a  method  for  specifying  a  time  window  on  goals 
and  activities,  for  external  events  and  their  time  of  occurrence,  and  for  delayed  events  caused 
some  time  after  a  planned  action.  NONLIN+  (Tate  &  Whiter,  1984)  added  the  capability  of 
representing  multiple  limited  resources  and  making  selection  from  appropriate  activities  on  the 
basis  of  reducing  some  overall  computed  “preference”  between  them.  The  definition  of  shared 
objects  as  resources  and  the  declaration  of  the  use  of  such  resources  in  operators  was  also  pro¬ 
vided  in  SIPE  (Wilkins,  1983). 

Airplan  (Masui  et  al,  1983)  maintained  information  on  the  operators  so  as  to  be  able  to  reason 
about  time  intervals  and  about  how  concurrent  actions  in  these  intervals  could  interact.  O-Plan 
(Bell  &  Tate,  1985)  uniformly  represents  time  constraints  (and  resource  usage)  by  a  numeric 
(min,  max)  pair  which  bound  the  actual  values  for  activity  duration,  activity  start  and  finish 
times,  and  delays  between  activities.  The  actual  values  may  be  uncertain  for  various  reasons, 
such  as  the  plan  being  at  a  high  abstraction  level,  not  having  chosen  values  for  objects  referred  to 
in  the  plan  or  uncertainty  in  modelling  the  domain.  Constraints  can  be  stated  on  the  time  and 
resource  values  which  may  lead  to  the  planner  finding  that  some  plans  in  its  search  space  are 
invalid.  EXCALIBUR  (Drabble,  1988)  allowed  for  planning  in  the  face  of  external  continuous 
processes  which  were  modelled  qualitatively. 

As  the  domains  being  considered  by  planning  systems  have  become  more  dynamic  (see  next  sec¬ 
tion)  information  has  also  been  placed  on  operators  to  allow  for  execution  time  monitoring 
(Firby,  1989)  ,  for  bounded  computation  or  real-time  scheduling  needs  (Kaelbling,  1987;  Hendler, 
1989),  to  allow  for  different  add-  and  delete-lists  depending  on  execution  time  success  or  failure, 
and  to  predict  the  probability  of  success  of  an  operator  (Miller  et.al.,  1985)  (See  Figure  5). 


Figure  5  Goes  around  here. 
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PICKUP-NEAREST  (LOC.X) 


\ 


FILTER  CONDITION: 

For-all  (X,Y)  DIST(LOC,X)  <  DIST(LOC.Y) 
&  OBTAINABLE(X) 

PRECONDITION:  ONTABLE(X)  A 

HANDEMPTY  A 
CLEAR(X) 

MONITOR:  (NEW-OBJ-MON 
Queue-length  1 
Run-time  75 
Run-every  500 
Reports: 

if  DiST(LOC.newobj)  <  DIST(LOC.X) 
then  UPDATE (X  =  NewObj) 
(PRESERVE-OBJ-MON 
Queue-length  1 
Run-time  25 
Run-every  250 
Reports: 

if  NOT(PRESERVE(x))  then  FAILURE) 
STEPS:  OPERATOR-MOVE-TO(X) 

LIFT(X) 

FAILURE-DEL:  (KNOWN  (LOC  X)) 
FAILURE-ADD:  (Achieve 
(PICKUP-NEAREST(CurLoc.X))) 
SUCCESS-ADD:  AT  (X) 

HOLDING(X) 

SUCCESS-DEL:  OBTAIN ABLE(X) 
ONTABLE(X) 

HANDEMPTY 

CLEAR(X) 

PROBABILITY:  .72 

RESOURCES:  TIME(DIST(LOC,X)  AVG-VEL) 
Consumes(Arm) 


Figure  5:  An  operator  for  a  (hypothetical)  modern  planning  system. 

The  operator  is  annotated  to  include  preconditions,  filter  conditions,  execution  time  mon¬ 
itoring  (with  real-time  scheduling  constraints),  steps  to  be  accomplished,  add-  and 
delete-lists  for  success  and  failure,  resource  usage  information,  and  a  success  probability. 


6.  Planning  and  Execution 

Most  of  the  systems  described  so  far  have  assumed  that  the  planner  is  possessed  of  complete 
knowledge  of  the  current  state  of  the  world  and  the  cause-and-effect  relationships  that  govern 
changes  in  that  world.  Clearly  this  approach  is  in  need  of  revision  in  situations  where  separate 
execution  cannot  be  guaranteed  to  succeed.  This  can  occur  when  agents  outside  of  the  control  of 
the  planner  may  cause  changes  in  the  environment  or  where  the  planner  may  be  uncertain  of 
information  that  can  only  be  ascertained  while  the  plan  is  being  run. 

For  example,  the  STRIPS  system  (Fikes  et.  al.,  1972a)  was  used  to  plan  the  motion  of  a  robot 
called  Skakcy  and  to  control  it  as  it  pushed  a  set  of  boxes  through  a  number  of  interconnecting 
rooms.  A  well  known  SRI  film  showed  Shakey  following  a  STRIPS  generated  plan  using  an  execu¬ 
tion  monitor  called  PLANEX.  Charley  Rosen,  the  SRI  AI  Lab  founder,  dressed  in  a  sinister 
cloak,  appeared  and  disrupted  the  position  of  the  boxes  during  execution.  PLANEX  was  able  to 
make  use  of  information  maintained  by  STRIPS  to  recover. 

This  approach  to  dealing  with  failure  when  it  arises  has  come  to  be  known  as  “replanning,”  and 
is  typically  assumed  to  occur  when  a  planner  recognizes  a  mismatch  between  the  expected  and 
actual  state  of  the  world.  Hayes  (1975)  proposed  that  the  subgoal  trees  and  decision  graphs  used 
in  the  formation  of  the  plan  could  be  used  to  guide  replanning.  Daniel  (1983)  explored  the  use  of 
a  similar  mechanism  in  a  partial  order  planner.  The  PRIAR  system  (Kambhampati  1989;  Kam- 
bhampati  &  Hendler,  1989)  uses  a  well-defined  subset  of  this  information,  known  as  the  valida¬ 
tion  structure,  to  provide  such  guidance.  (A  formal  treatment  of  replanning  can  be  found  in  Mor- 
genstern  1987). 

An  alternative  to  replanning  is  to  actually  expect  potential  failures  and  to  plan  for  them.  This 
can  involve  planning  for  expected  possibilities,  such  as  waiting  for  a  light  to  turn  green  before 
crossing  a  street  (Drummond,  1986),  or  it  can  involve  scheduling  monitoring  tasks  —  tests  to  be 
run  at  execution  time,  associated  with  fixes  to  be  used  in  the  case  of  failed  tests  (cf.  Doyle,  et.  al. 
1986).  An  example  of  this  (Kaelbling,  1987)  would  be  having  a  robot  check  to  see  if  two  walls 
were  equidistant  at  some  point  during  the  traversal  of  a  hallway.  If  not,  the  robot  could  achieve 
this  equidistance,  and  then  continue.  Schoppers  (1987)  has  proposed  taking  disjunctive  planning 
to  an  extreme  by  generating  “universal”  plans  —  plans  with  conditional  tests  to  deal  with  all 
possible  execution-time  contigencies. 

Another  approach  to  handling  change  in  the  environment  is  to  provide  a  different  sort  of  integra¬ 
tion  between  generation  and  execution  of  plans.  McDermott’s  (1978)  NASL  system,  for  example, 
interleaved  plan  generation  and  execution  by  choosing  one  step  at  a  time  and  executing  it.  This 
made  the  planner  more  susceptible  to  errors  caused  by  interactions,  but  made  it  less  susceptible  to 
errors  caused  by  change  in  the  environment.  This  works  well  in  environments  where  a  small 
amount  of  change  may  occur,  but  is  insufficient  in  domains  in  which  rapid  reaction  is  needed  (for 
example  the  Traffic  World  (Hendler  &  Sanborn,  1987)  shown  in  Figure  6).  This  approach  has 
been  extended  by  Drummond  (1989),  where  Situated  Control  Rules  are  used  to  inform  an  indepen¬ 
dently  competent  execution  system  —  the  execution  system  can,  if  necessary,  act  without  a  plan; 
the  plan  simply  serves  to  increase  the  system’s  goal-achieving  abilities. 


Figure  6  goes  about  here. 


Much  recent  work  has  dealt  with  designing  mechanisms  which  can  handle  rapidly  changing 
environments.  Most  of  this  work  achieves  responsiveness  by  giving  up  complex  planning  for  shal¬ 
low  planning  or  by  planning  using  tightly  coupled  sensing  and  action  (Rosenschein,  1982;  Chap¬ 
man  &  Agre,  1987;  Sanborn  &  Hendler,  1988).  Some  of  this  work,  however,  has  dealt  directly 
with  the  issues  of  how  to  map  from  planning  to  reaction.  Firby  (1989)  proposes  “reactive  action 
packages”  which  essentially  replace  the  operators  in  the  system  with  procedures  that  include  a 
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Figure  8:  The  Traffic  World  (Hendler  &  Sanborn,  1987). 

The  Traffic  World  consists  of  a  straight  stretch  of  “4-lane  highway”  along  which  “cars” 
travel  at  various  speeds.  There  is  a  much  slower  moving  “robot”  at  one  end  of  the  high¬ 
way,  who  is  trying  to  get  across  to  the  opposite  side.  Cars  changes  lanes  and  speeds  as 
they  proceed. 


reactive  component.  Kaelbling’s  and  Rosenschein’s  (1988)  GAPPS  system  is  a  compiler  that 
translates  constraint  expressions  into  directly  executable  circuits  for  use  in  robotic  control  sys¬ 
tems.  Georgeff  and  Lansky  (1987)  describe  the  use  of  a  meta-reasoning  system  which  can  choose 
from  a  variety  of  execution  time  options  based  on  the  goals  being  pursued  by  the  system. 
Ambros-Ingerson  and  Steel  (1987)  propose  an  approach  to  integrating  planning  and  execution  for 
changing  domains  using  an  agenda-driven  control  structure  in  which  actions  which  are  serially 
initiated  can  run  concurrently,  with  information  acquiring  actions  (for  monitoring  the  environ¬ 
ment)  included.  In  addition,  an  effort  is  being  made  to  extend  temporal  representations  to  handle 
simultaneously  occuring  events  and  event  interactions.  The  use  of  these  extended  representations 
for  planning  is  discussed  by  Pednault  (1987). 

In  addition,  it  now  appears  that  an  important  part  of  planning  in  dynamic  domains  involves 
making  tradeoffs  —  specifically  trading  precision  in  decision  making  for  time  in  responding  to 
events.  In  the  last  few  years,  a  number  of  researchers  have  attempted  to  improve  the  responsive¬ 
ness  of  planning  systems  operating  in  dynamic  domains  by  directly  reasoning  about  the  utility  of 
planning  (Dean,  1987;  Horvitz,  1988;  Russell  &  Wefald,  1989).  This  work  has  involved  an  exami¬ 
nation  both  of  reasoning  about  these  trade-offs  during  plan  generation  (Kanazawa  and  Dean, 
1989;  Heckerman  et  al,  1989)  or  during  execution  (Boddy  and  Dean,  1989;  Horvitz  et  al,  1989). 


7.  Learning  and  Memory 

The  concentration  in  most  of  the  work  on  planning  has  been  on  generating  plans  from  scratch, 
not  learning  from  experience.  Thus,  much  of  the  classical  work  in  planning  has  been  “ahistoric” 
—  that  is,  asked  to  solve  the  same  problem  again  the  planner  performs  no  better  than  it  did  the 
first  time.  Recently,  due  both  to  the  gains  being  made  in  machine  learning  and  the  new  work  on 
case-based  reasoning,  designing  planning  systems  which  learn  from  experience  has  been  an  option. 

The  earliest  approach  to  learning  in  plans  was  the  MACROPs  (for  macro-operators)  work  of 
(Fikes  et  al  1982b)  which  extended  STRIPS  to  do  some  limited  learning  from  its  failures.  When  a 
portion  of  a  plan  was  found  to  have  succeeded,  the  entire  set  of  operators  could  be  turned  into  a 
single  operator  whose  preconditions  and  effects  were  identical  with  the  preconditions  and  effects 
of  the  operator  sequence.  The  operators  were  generalized  by  using  variables  to  replace  the  con¬ 
stants  found  in  the  specific  solution  from  which  the  new  operator  had  been  derived.  Minton’s 
(1985)  Prodigy/EBL  system  used  a  similar  approach,  but  applied  an  explanation-based  learning 
algorithm  to  provide  more  accurate  generalizations. 

Case-based  reasoning  approaches  to  planning  have  also  been  attempted.  In  these  systems  an  old 
plan  is  chosen  and  modified  to  run  in  the  new  situation.  Many  of  these  systems  have  concen¬ 
trated  on  guiding  the  search  for  the  old  plan  (e.g.  JULIA,  Kolodner  1987;  CHEF,  Hammond 
1986)  and  then  use  a  fairly  simple  mapping  to  produce  the  new  plan.  PLEXUS  (Alterman,  1988) 
used  information  about  the  new  context  to  guide  the  reuse  of  an  existing  plan.  The  PRIAR  reuse 
system  (Kambhampati,  1989)  applied  a  case-based  approach  in  the  classical  planning  framework. 
PRIAR,  an  extension  to  NONLIN  (Tate,  1977),  allowed  the  planner  to  annotate  plans  being 
created  with  information  about  the  dependency  structure  between  operators.  This  information 
was  then  used  to  guide  retrieval,  reuse,  and  replanning. 

8.  Assorted  Shorts 

As  well  as  the  issues  discussed  above,  many  other  issues  arise  in  the  design  of  planning  systems. 
Some  examples  include: 

Planning  with  Conditionals  and  Iterators 

Most  of  the  search  space  control  techniques,  goal  ordering  and  interaction  correction 
mechanisms  developed  in  Al  planners  to  date  have  been  oriented  towards  the  generation  of 
plans  which  are  fully  or  partially  ordered  sequences  of  primitive  actions.  However,  there 
has  been  some  effort  on  the  generation  of  plans  which  contain  conditionals  (“if  ...  then  ... 
else  ...")  and  iterators  ("repeat  ...  until  ...*).  Conditionals  were  handled  in  WARPLAN-C 
(Warren,  1976)  by  branching  the  plan  at  the  conditional  and  performing  separate  planning 
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on  the  two  branches  using  the  assumption  that  the  statement  in  the  conditional  was  true  in 
one  branch  and  false  in  the  other.  This  led  to  a  case  analysis  of  the  separate  branches  to 
produce  a  plan  that  was  tree-structured.  Drummond  (1985)  designed  an  extension  to  pro¬ 
cedural  nets  allowing  for  disjunction  and  iteration  and  usable  for  predicting  the  behavior  of 
the  system  where  these  were  present. 

Uncertainty  in  Planning 

One  source  of  execution  time  problems  is  that  of  uncertainty  which  exists  during  plan  gen¬ 
eration.  If  the  planner  cannot  model  the  real  world  with  complete  information,  but  instead 
uses  some  sort  of  probability-based  model,  it  must  deal  with  low  probability  events  that 
may  occur  during  execution.  (This  problem  arises  in  a  very  significant  way  for  systems 
which  use  real  sensors  to  perceive  the  world).  Work  using  probability  estimates  during  plan 
generation  includes  Dean  et.  al  (1989)  and  Dean  &  Kanazawa  (1989).  Segre  (1988)  examines 
the  issue  of  execution  time  failures  of  plans  based  on  uncertain  information. 

Distributed  Planning 

Some  systems  have  been  exploring  distributing  sources  of  problem  solving  expertise  or 
knowledge  during  planning.  They  allow  fully  distributed  planning  with  the  sub-problems 
being  passed  between  specialized  planning  experts.  The  use  of  the  experts,  or  the  behaviors 
they  may  perform,  are  controlled  through  a  centralized  blackboard  and  executive  (with  a 
system  rather  like  priority  scheduling  of  parallel  processes)  or  may  be  controlled  in  a  more 
distributed  fashion  via  pairwise  negotiation.  Examples  of  relevant  work  include  Smith’s 
(1977)  Contract  Net,  Corkill  (1979),  Kornfeld  (1979),  Konolige  and  Nilsson  (1980)  Georgeff 
(1982),  and  Corkill  and  Lesser  (1983). 

9.  Recommended  Reading 

A  good  and  reasonably  up-to-date  account  of  AI  planning  techniques  and  systems  is  given  in 
Charniak  and  McDermott’s  (1985)  Introduction  to  Artificial  Intelligence  textbook.  In  particular, 
chapter  9  and  sections  of  chapters  5  and  7  are  relevant.  Somewhat  earlier  material  is  provided  in 
Elaine  Rich’s  (1983)  Artificial  Intelligence  textbook.  Nilsson’s  (1980)  book  on  the  Principles  of 
Artificial  Intelligence  provides  a  uniform  treatment  of  planning  techniques  available  up  to  the 
time  it  was  published.  There  are  several  useful  summaries  of  early  AI  planning  work  in  the 
Handbook  of  Artificial  Intelligence  (Barr  &  Feigenbuam,  1981)  volume  I  section  II.D  and  volume 
III  sections  XI.B,  XI. C  and  XV.  A  collection  of  recent  papers  concerning  planning  can  be  found  in 
Georgeff  and  Lansky’s  (1987)  Reasoning  about  Actions  and  Plans.  A  collection  of  previously  pub¬ 
lished  papers  on  planning  and  reasoning  about  actions  will  soon  be  available  as  Readings  in  Plan¬ 
ning  (Allen  et.  al.)  ,  expected  to  be  published  in  the  Spring  of  1990. 

Allen,  J.,  Hendler,  J.  and  Tate,  A.  (eds.)  (1990)  “Readings  in  Planning,”  Los  Altos,  Ca:  Morgan- 
Kaufmann.  Barr,  A.  and  Feigenbaum,  E.A.  (1981)  "The  Handbook  of  Artificial  Intelli¬ 

gence*  Los  Angeles,  Ca:  William  Kaufmann. 

Charniak.  E.  and  McDermott,  D.V.  (1985)  "Introduction  to  Artificial  Intelligence,"  Reading,  Ma: 
Addison-Wesley. 

Georgeff,  M.  and  Lansky,  A.  (eds.)  (1987)  “Reasoning  about  actions  and  plans,”  Los  Altos,  Ca. 
Morgan-Kaufmann 

Nilsson,  N.J.  (1980)  "Principles  of  Artificial  Intelligence",  Palo  Alto,  Ca:  Tioga  Press. 

Rich,  E.  (1983)  "Artificial  Intelligence",  McGraw-Hill,  New  York. 

10.  Summary 

Planning  systems  have  been  an  active  research  topic  within  Artificial  Intelligence  for  nearly  thirty 
years.  There  have  been  a  number  of  techniques  developed  during  that  period  which  still  form  an 
essential  part  of  many  of  today’s  AI  planning  systems.  In  this  paper  we  have  tried  to  provide  a 
broad  coverage  of  the  major  ideas  in  the  field  of  AI  planning,  and  have  attempted  to  show  the 
direction  in  which  current  research  is  going.  Such  a  task  is  never-ending,  and  any  finite 
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document  must  per  force  be  incomplete.  We  have  provided  references  to  connect  each  idea  to  the 
appropriate  literature  and  to  allow  readers  immediate  access  to  the  papers  most  relevant  to  their 
research  or  applications  work. 
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